Towards performing everyday manipulation activities

This article investigates fundamental issues in scaling autonomous personal robots towards open-ended sets of everyday manipulation tasks which involve high complexity and vague job specifications. To achieve this, we propose a control architecture that synergetically integrates some of the most promising artificial intelligence (AI) methods that we consider as necessary for the performance of everyday manipulation tasks in human living environments: deep representations, probabilistic first-order learning and reasoning, and transformational planning of reactive behavior — all of which are integrated in a coherent high-level robot control system: COGITO. We demonstrate the strengths of this combination of methods by realizing, as a proof of concept, an autonomous personal robot capable of setting a table efficiently using instructions from the world wide web. To do so, the robot translates instructions into executable robot plans, debugs its plan to eliminate behavior flaws caused by missing pieces of information and ambiguities in the instructions, optimizes its plan by revising the course of activity, and infers the most likely job from vague job description using probabilistic reasoning.